Important Deviations from Study Protocol

To the Editor: As authors of a guideline and a systematic review on interventions for preventing and reducing physical restraints (PR) in long-term geriatric care, we eagerly awaited the publication by Koczy and colleagues on the Redufix trial that had been completed 5 years before. Since then, without public access to the final results, the intervention has been promoted throughout Germany as a successful method of reducing physical restraints, with a number of follow-up projects supported by the German Federal Ministries. We read the article with great interest but find it difficult to judge the intervention’s efficacy because a number of questions arise concerning methodological and reporting issues. The study has not been registered in a public trials registry. Thus, the reader cannot assess the investigators’ adherence to their original planning, but Koczy and colleagues have published a study protocol in German. Comparison of the protocol and the Journal of the American Geriatrics Society (JAGS) article indicates several post hoc changes, most important the redefinition of the primary endpoint. The protocol defines the primary endpoints as ‘‘number of persons with PR and number of hours of application,’’ whereas in the JAGS article, the main outcome is ‘‘complete cessation of PR use in residents 91 to 93 days after the start of the intervention,’’ which means that the planned end point, ‘‘reduction in the number of residents with PR’’, was changed to ‘‘increase in the number of residents without PR.’’ This reflects a completely different perspective on the results, because different populations are considered. Originally, the authors assumed a relative risk reduction of 50% (5% of residents with PR in the control group (CG) and 2.5% in the intervention group (IG)) referring to the use of PR in the whole population of residents. Using the altered end point and the redefined reference population, the results show 83.2% of residents with PR in the IG, compared with 91.2% in the CG (a relative risk reduction of 9%). The assumed and the actual results cannot be compared. As in the JAGS article, there is no information on the use of PR in the whole population, so it is not possible to judge on the magnitude of the effect. Based on the original end point, the required sample described in the protocol is 54 clusters with 100 residents each. Surprisingly, the JAGS article does not report an adapted power calculation despite the altered end point and the smaller sample size of only 45 clusters. The definition of PR also differs from the protocol. In the JAGS article, only belts and fixed tables were considered to be PR, whereas the protocol also includes other measures, including braked wheelchairs near a table and ‘‘protective sheets.’’ In our view, bedrails should not be excluded from PR, even if this is in accordance with other studies in the field. At least the number of residents with bedrails should be reported to judge possible unwanted effects of the intervention and a possible shift of PR toward bedrails. Unsatisfactorily, analyses have not taken the cluster design into account, indicating a unit-of-analysis bias and further deviation from the study protocol. We are not convinced by the rationale now given by Koczy and colleagues for the avoidance of cluster adjustment. It remains unclear how the 1.5% variance has been calculated. The pronounced sex distribution difference between the study groups further points out the necessity of cluster adjustment. The authors should have provided at least a clusteradjusted sensitivity analysis. Use of proper cluster adjustment would most likely have resulted in loss of statistical significance of the primary end point because the lower limit of the confidence interval is already at the brink (odds ratio52.16, 95% confidence interval51.05–4.46). Not only does the article’s internal validity remain difficult to assess, but its external validity must also be contested. The study is a further example of lack of reporting of details on the development and evaluation of complex interventions, including insufficient reporting of the underlying theory, modeling of components, piloting of feasibility and acceptability, and standardized introduction of the intervention throughout different centers. In conclusion, we would have expected the authors to adhere to the study protocol even if this would have led to a nonsignificant result. Rather than ‘‘fishing for P-values,’’ analyses of strengths and weaknesses of the program would have been of great significance even if not ‘‘statistically significant.’’ Considering the complexity and effort of the program, transparent and comprehensive reporting of the study results with adequate statistical methods would have been an important step toward the development of future programs to reach the imperative goal of avoiding PR use in nursing home residents. Unfortunately, this is not the case.